Skip to content

TOMEE-4652 - roll back UserTransaction left over by a request - #2849

Open
jungm wants to merge 2 commits into
apache:mainfrom
jungm:claude/tomee-4652-fix-bc96d4
Open

TOMEE-4652 - roll back UserTransaction left over by a request#2849
jungm wants to merge 2 commits into
apache:mainfrom
jungm:claude/tomee-4652-fix-bc96d4

Conversation

@jungm

@jungm jungm commented Jul 23, 2026

Copy link
Copy Markdown
Member

TOMEE-4652

A servlet or JSP that leaves a bean-managed UserTransaction incomplete leaks that transaction to the next request served on the same pooled Tomcat exec thread. The victim request then sees a bogus transaction status — either missing an expected IllegalStateException or getting a NotSupportedException: Nested Transactions are not supported on its own begin(). Which tests fail depends on which request lands on which thread, which is why the Transactions 2.0 TCK web vehicles (servlet + JSP) fail non-deterministically.

Root cause

Geronimo's TransactionManagerImpl keeps the thread-to-transaction association (and the per-thread transaction timeout) in ThreadLocals that are only cleared by commit() / rollback(). EJBs are wrapped by container interceptors that restore the thread state at the end of the call; plain servlets have no equivalent, and OpenEJBValve's request-teardown finally block cleaned up only the security context. Since Tomcat pools its worker threads, the association survives into the next request.

Fix

  • TransactionCleanup (new) — rolls back and unassociates any transaction still active on the thread at request end, and resets the per-thread transaction timeout (which leaks the same way, since Geronimo only clears it inside begin()). If the rollback itself fails it falls back to suspend() so the association never survives the request.
  • Invoked from the request-teardown finally in OpenEJBValve (sync path) and OpenEJBSecurityListener.asyncExit() (async complete/error/timeout).
  • CoreUserTransaction.resetError(null) now remove()s the ERROR ThreadLocal instead of set(null), so pooled threads don't keep an empty entry pinned. Separate hygiene issue, not the TCK cause.

Testing

UserTransactionLeakTest forces two sequential requests onto a single exec thread (maxThreads=1) and asserts both actually shared the thread (so it can't pass vacuously), that the second request sees STATUS_NO_TRANSACTION, and that it can still run a transaction of its own.

Verified red/green: with the cleanup call removed the test fails with expected:<[STATUS_NO_TRANSACTION]> but was:<[leaked status 0]> and a follow-up NotSupportedException: Nested Transactions are not supported — matching the issue exactly; with the fix it passes. tomee-catalina and tomee-embedded suites are green.

Notes for reviewers

  • The full Jakarta Transactions 2.0 TCK was not run here. To confirm end to end, remove the three excluded areas from runner-standalone/exclusions/transactions.txt in the apache/tomee-tck harness and rerun the 49-test baseline.
  • Pre-existing failures unrelated to this change exist on main in StatefulBeanManagedTest, InterfaceTransactionTest, and TransactionPropagationTest (confirmed identical on a clean checkout).

🤖 Generated with Claude Code

A servlet or JSP that leaves a bean managed UserTransaction incomplete
leaks that transaction to the next request served on the same pooled
Tomcat exec thread. Geronimo's TransactionManagerImpl keeps the
thread-to-transaction association (and the per-thread transaction
timeout) in ThreadLocals that are only cleared on commit()/rollback().
EJBs are wrapped by container interceptors that restore the thread
state; plain servlets have no equivalent, and OpenEJBValve's finally
block cleaned up only the security context.

Add TransactionCleanup, invoked from the request teardown finally in
OpenEJBValve (sync path) and OpenEJBSecurityListener.asyncExit()
(async complete/error/timeout). It rolls back and unassociates any
dangling transaction and resets the per-thread transaction timeout,
which leaks the same way.

Also make CoreUserTransaction.resetError(null) remove() the ThreadLocal
instead of set(null) so pooled threads don't keep an empty entry pinned.

Adds UserTransactionLeakTest, which forces two sequential requests onto
one exec thread (maxThreads=1) and asserts the second sees no leaked
transaction.
@rzo1

rzo1 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Verified the premise against the Geronimo 4.0.0 sources — unassociate() and begin()
are the only places threadTx and transactionTimeoutMilliseconds get cleared, so a web
request really can strand both on a pooled exec thread. Rolling back at request teardown
is the right fix and matches TxBeanManaged.commit(). The single-exec-thread test is a
genuinely good reproduction, and the guard that fails loudly if thread reuse didn't happen
is the right instinct.

The placement rationale is wrong, though, and it should be corrected in the javadoc
because it changes what applications observe:

  • The description says the valve runs after ServletRequestListener#requestDestroyed.
    It's the opposite. I disassembled tomcat-catalina 11.0.23: StandardContextValve has no
    fireRequest* call at all. StandardHostValve.invoke fires fireRequestInitEvent at
    offset 72, invokes the Context pipeline (where OpenEJBValve and therefore clean()
    live) at offset 125, and fires fireRequestDestroyEvent only at offset 327 — after the
    pipeline returns.

    So an application ServletRequestListener implementing a tx-per-request pattern and
    committing in requestDestroyed now finds the transaction already rolled back, gets a
    WARN on every single request, and an IllegalStateException from its own commit().
    Filter- and servlet-based patterns are unaffected, so this doesn't invalidate the fix —
    but pre-empting requestDestroyed is a real behavioural change and belongs in the
    javadoc.

  • Third uncovered path, more concrete than the async ones: StandardHostValve.invoke
    calls throwable()/status() at offsets 195/298/307, after the pipeline, and
    custom() dispatches the error page through ApplicationDispatcher.include(), not a
    Pipeline. So <error-page> servlets and JSPs run after clean() — they leak exactly as
    before this PR, and they now also run with the request's transaction already rolled back
    and the caller identity already cleared. If you want that covered,
    OpenEJBSecurityListener.RequestCapturer on the Host pipeline
    (TomcatWebAppBuilder:317) wraps all of it.

Smaller:

  • In OpenEJBValve, TransactionCleanup.clean() is skipped if listener.exit() throws.
    The async path already gets this right with a nested finally — please mirror it here.
  • setTransactionTimeout(0) pins the very ThreadLocal entry that the
    CoreUserTransaction.resetError hunk in this same PR argues against pinning. Worth a
    comment explaining why the tradeoff differs, or reading the timeout first and only
    resetting when non-zero.
  • The timeout reset is never asserted. Leaker sets setTransactionTimeout(120) with a
    comment saying it must not leak, and then nothing checks it — the branch is untested.
  • Unconditional rollback() logs an ERROR when the leftover transaction is no longer
    rollback-able (already rolled back / marked for rollback by a timeout reaper). Check
    getStatus() against STATUS_ROLLEDBACK/STATUS_ROLLING_BACK first, or log at debug.

Merge precondition rather than a code comment: please re-run the Jakarta Transactions TCK
and drop the corresponding exclusions in apache/tomee-tck with this, since that's the
harness that surfaced it.

Review feedback on the placement, which was both documented backwards
and covering less than it claimed.

StandardContextValve has no fireRequest* call at all; StandardHostValve
fires requestInitEvent, invokes the Context pipeline, and only then runs
throwable()/status() and fires requestDestroyEvent. So cleaning up from
OpenEJBValve (Context pipeline) ran *before* requestDestroyed rather
than after it, pre-empting applications that complete their transaction
there, and left <error-page> servlets and JSPs leaking exactly as before
since they are dispatched through an include rather than a Pipeline.

Move the call to OpenEJBSecurityListener.RequestCapturer, which is on
the Host pipeline and wraps all of StandardHostValve#invoke, and correct
the javadoc to describe the real ordering. This also drops the call from
OpenEJBValve entirely, so the reported "skipped when listener.exit()
throws" hole goes away with it.

Also from review:
- don't log an ERROR when the leftover transaction was already rolled
  back or is rolling back; unassociate it at debug instead
- explain in resetTimeout() why pinning the timeout ThreadLocal is the
  right tradeoff here even though resetError() avoids pinning: the
  timeout cannot be read back, and a null value cannot pin a classloader
- actually assert the timeout does not leak. The Leaker set a 120s
  timeout with a comment saying it must not leak and nothing checked it.
  Committer now reads the effective timeout of its own transaction and
  the test distinguishes the leaked 120s from the 600s default. Verified
  this assertion fails when the cleanup is removed.
@jungm

jungm commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

Thanks — the ordering correction was right, and I verified it independently before acting on it (javap on tomcat-catalina 11.0.23: StandardContextValve has zero fireRequest* calls; StandardHostValve.invoke has fireRequestInitEvent at 72, the Context pipeline at 113, throwable()/status() at 195/298/307, and fireRequestDestroyEvent at 327).

Rather than only fix the javadoc, I moved the cleanup to OpenEJBSecurityListener.RequestCapturer on the Host pipeline, which wraps all of StandardHostValve#invoke. That covers requestDestroyed, the throwable()/status() paths, and <error-page> servlets/JSPs, and it stops pre-empting applications that complete their transaction in requestDestroyed. The javadoc now describes the real ordering. This also removes the OpenEJBValve call site entirely, so the "skipped if listener.exit() throws" hole goes away with it.

Also addressed:

  • Already-rolled-back status: rollback() now checks STATUS_ROLLEDBACK/STATUS_ROLLING_BACK and logs at debug in that case, warning otherwise.
  • Timeout reset pinning: kept unconditional, with a comment explaining why the tradeoff differs from resetError. Reading the timeout first isn't available — Geronimo exposes no getter, only the package-private getTransactionTimeoutMilliseconds(long). And setTransactionTimeout(0) stores a null value, which can't pin a webapp classloader the way a stored exception's stack trace can.
  • Timeout never asserted: you were right that the branch was untested. Committer now reads the effective timeout of its own transaction and the test distinguishes the leaked 120s from the 600s default. Worth noting the first version of this assertion was a false green — the field is an absolute deadline (duration + currentTime), not a duration, so comparing it to 120000 could never match. Fixed to compare the remaining duration, and confirmed it fails when the cleanup is removed.

On the merge precondition: I have not run the Jakarta Transactions TCK, so the exclusions in apache/tomee-tck are untouched and this shouldn't merge on my testing alone. tomee-catalina and tomee-embedded suites are green.

🤖 Addressed by Claude Code

@rzo1

rzo1 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Please run a CI full build for this.

@jungm

jungm commented Jul 31, 2026

Copy link
Copy Markdown
Member Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants